An original image obtained by an image pickup apparatus is deteriorated due to an image blurring component contained therein which is caused by adverse effects that occur in an image pickup optical system. These adverse effects include various types of aberrations, such as spherical aberration, coma aberration, curvature of field, and astigmatism, and light diffraction. The presence of such an image blurring component caused by these types of aberrations means that when a light beam emitted from one point does not suffer from any aberration or diffraction, the light beam fails to converge to another point, being minutely spread around the another point. Such a minutely-spread distribution is represented by point spread function (PSF).
An optical transfer function (an OTF) obtained by the Fourier transform of a PSF is frequency component information on an aberration and represented by a complex number. The absolute value of an OTF, i.e., an amplitude component, is referred to as a “modulation transfer function” (an MTF), and a phase component is referred to as a “phase transfer function” (a PTF), respectively. A modulation component MTF and a phase component PTF are an amplitude component of image deterioration caused by an aberration, and frequency characteristics of a phase component, respectively, and represented by the following expression with the phase component being defined as a phase angle.PTF=tan−1(Im(OTF)/Re(OTF))
In this expression, symbols Re (OTF) and Im (OTF) represent the real part and the imaginary part of an OTF, respectively. As described above, because an OTF of an image pickup optical system deteriorates an amplitude component and a phase component of an image, each point of an object is, as seen in the case of a coma aberration, asymmetrically blurring in the deteriorated image. A chromatic aberration of magnification occurs in a situation in which a position where an image is formed is shifted due to a difference in the formation ratio depending on each light wave length and the formed image is obtained as an RGB color component, for example, according to spectral characteristics of the image pickup apparatus. This means that imaging positions of R, G, and B components are shifted to each other, leading to an imaging position shift in each color component depending on a wavelength, that is, a spread of an image due to a phase shift.
As a method of correcting a deterioration of an amplitude component MTF and a phase component PTF, the correction with the use of information on an OTF of the image pickup optical system is known. This method is generally called image restoration or image reconstruction. Accordingly, processing in which a deteriorated shot image is corrected by using information on an OTF of the image pickup optical system is hereinafter referred to as “image restoration processing”. As one of the image restoration methods, the method of convoluting an image restoration filter with reverse characteristics of an optical transfer function (OTF) with respect to an input image, which is described later, is known.
The effective use of a restored image requires obtaining more accurate information on an OTF of the image pickup optical system. An OTF of a typical image pickup optical system is greatly different depending on an image height (a position in an image). Moreover, since an OTF is two-dimensional data represented by a complex number, the OTF has real and imaginary parts. When the image restoration processing is to be performed for a colored image which includes three color components, that is, red (R), green (G), and blue (B), OTF data with a single image height is represented by the following expression: tap number in a vertical direction×tap number in a horizontal direction×2 (real part/imaginary part)×3 (RGB). In this expression, “tap number” means vertical and horizontal sizes of the OTF data. The retention of these tap numbers for all items of an image pickup condition including an image height, an F number (an aperture value), a zoom (a focal length), and an object distance results in a massive data volume. In order to change restoration characteristics depending on a position in an image, it is desirable to perform the image restoration processing not in a frequency space in a batch manner, but in a real space while switching a restoration filter.
One cause for image blurring is a diffraction phenomenon of light dependent on an F number of the image pickup optical system. In FIG. 9, which illustrates a diffraction limit curve, the horizontal axis and the vertical axis indicate a spatial frequency and a MTF, respectively. As illustrated in FIG. 9, the darker color the F number indicates, to the lower frequency side the cutoff frequency is shifted. For instance, the Nyquist frequency of an image pickup element with a pixel size of 4 μm is 125 lines/mm. Accordingly, when an F number, for example F2.8, indicates a bright color, the degree of adverse effects caused by such a diffraction phenomenon is small. By contrast, when an F number, for example F16 or F32, indicates a dark color, the degree of such adverse effects is large. Since a diffraction phenomenon can be represented by an OTF or a PSF as in the case of an aberration, the image restoration processing described above can correct a blurring due to diffraction.
PTL 1 discloses a method of performing the image restoration processing according to various image pickup conditions of an image pickup apparatus while retaining an OTF for use in image restoration which is converted to a coefficient.